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This is in response to the appeal brief filed 10/02/2008 appealing from the Office action 
mailed 03/25/2008. 
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(1) Real Party in Interest 

A statement identifying by name the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

The examiner is not aware of any related appeals, interferences, or judicial 
proceedings which will directly affect or be directly affected by or have a bearing on the 
Board's decision in the pending appeal. 

(3) Status of Claims 

The statement of the status of claims contained in the brief is correct. 

(4) Status of Amendments After Final 

The appellant's statement of the status of amendments after final rejection 
contained in the brief is correct. 

(5) Summary of Claimed Subject Matter 

The summary of claimed subject matter contained in the brief is correct. 

(6) Grounds of Rejection to be Reviewed on Appeal 

The appellant's statement of the grounds of rejection to be reviewed on appeal is 
correct. 

(7) Claims Appendix 

The copy of the appealed claims contained in the Appendix to the brief is correct. 

(8) Evidence Relied Upon 

WO 03/041000, "Image Capture and Identification System and Process", Boncyk et al, 
05/15/2003 
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2003/0164819 Waibel 09-2003 

2002/0049728 Kaku 04-2002 

Haynes Repair Manuals, 03/25/2002, retrieved 12/04/2008 from www.archive.org, url 
http://web.archive.Org/web/20020325061432/http://haynes.com/ 
(9) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 
Claims 1-8, 11-15, 17-19, and 21-28 are rejected under 35 U.S.C. 102(e) as 
being anticipated by Boncyk et al. (WO 03/041000). 
Regarding claim 1, Boncyk et al. discloses: 

A system for image-based information retrieval from search 
engines, characterized by a) a terminal with a built-in camera that is 
connected to a remote data transmission network (8:22-10:5); b) a server 
computer on which an object recognition program is running, which 
analyses images sent to it and provides them with a symbolic indexing 
(37:30-38:1 and 18:6-10); c) a search engine that uses the indexed image 
to find information about the image and sends it back to the terminal 
(37:30-38:1 and 18:6-10). 
Regarding claim 2, see the rejection of claim 1 and note that Boncyk et al. further 
discloses: 

The system as described under 1) that is designed for mobile 
telephones or portable computers that have a built-in camera (6:7-19). 
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Regarding claim 3, see the rejection of claim 1 and note that Boncyk et al. further 
discloses: 

A city or museum guide that uses the system described under 2) to 
provide a user with information about objects of which he or she has 
previously taken a picture (2:5-7). 
Regarding claim 4, see the rejection of claim 3 and note that Boncyk et al. further 
discloses: 

The system as described under 3) in which positioning information 
is also used to appropriately limit the image recognition system (27:36- 
28:14). 

Regarding claim 5, see the rejection of claim 2 and note that Boncyk et al. further 
discloses: 

The system as described under 2) that provides product information 
about products that have been previously photographed with the mobile 
camera (27:36-28:14). 
Regarding claim 6, see the rejection of claim 2 and note that Boncyk et al. further 
discloses: 

The system as described under 2) in which the image recognition 
system is also able to recognize text characters or symbols (27:36-28:14). 
Regarding claim 7, see the rejection of claim 2 and note that Boncyk et al. further 
discloses: 
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The system as described under 2) in which the system is, in 
particular, able to recognize faces (27:7-9). 
Regarding claim 8, see the rejection of claim 2 and note that Boncyk et al. further 
discloses: 

The system as described under 2) that is used to provide the user 
with additional information about advertising billboards (39:18-24). 
Regarding claim 11, Boncyk et al. discloses: 

A computer implemented system for image-based searching, 
comprising: a computer server, communicatively coupled with a network, 
that receives an input image from a user device communicatively coupled 
with the network (8:22-10:5); an image recognition system executed by 
the computer server and adapted to: determine a plurality of graphical 
attributes of the input image (37:30-38:1 and 18:6-10); match the input 
image to a reference image from a plurality of reference images stored in 
the storage medium (1 5:1-1 5:35), based on the plurality of graphical 
attributes of the input image and the reference images, each of the 
reference images having an associated symbolic identifier (37:30-38:1); 
and associate a symbolic identifier to the input image based on the 
symbolic identifier associated with the matching reference image (37:30- 
38:1); a search engine executed by the computer server and adapted to 
receive a query and to retrieve a set of search results associated with the 
query (9:40-10:5, URL); and a query processing system executed by the 
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computer server and adapted to: receive the symbolic identifier of the 
input image from the image recognition system (9:40-10:5, URL); provide 
the symbolic identifier to the search engine as a query, and to receive a 
set of search results associated with the symbolic identifier (URL lookup 
9:40-10:5); and transmit, via the network, a plurality of the search results 
to the user device (9:40-10:5 and 25:1-40). 
Regarding claim 12, see the rejection of claim 1 1 and note that Boncyk et al. 
further discloses: 

User device comprises a mobile telephone having an integrated 
camera (6:7-19). 

Regarding claim 13, see the rejection of claim 1 1 and note that Boncyk et al. 
further discloses: 

Server receives a geographic location of the user device in 
association with the input image (27:36-28:14); and the image recognition 
system is further adapted to match the input image to a reference image 
from the plurality of reference images based on the geographic location of 
the user device (27:36-28:14). 
Regarding claim 14, see the rejection of claim 1 1 and note that Boncyk et al. 
further discloses: 

Image recognition system further includes a character recognition 
system (27:36-28:14). 
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Regarding claim 15, see the rejection of claim 1 1 and note that Boncyk et al. 
further discloses: 

Image recognition system further includes a facial recognition 
system (27:7-9). 
Regarding claim 17, Boncyk et al. discloses: 

A computer implemented method for image-based searching, 
comprising: receiving at a computer server, an input image from a user 
device remotely located from the server (8:22-10:5); providing from the 
computer server the input image to an image recognition system (37:30- 
38:1 and 18:6-10); receiving at the computer server from the image 
recognition system a symbolic identifier associated with the input image 
(9:40-10:5, URL); providing from the computer server the symbolic 
identifier to a search engine as a query (9:40-10:5, URL); receiving at the 
computer server from the search engine a set of search results associated 
with the symbolic identifier (URL lookup 9:40-10:5); and transmitting from 
the computer server a plurality of the search results to the user device 
(URL lookup 9:40-10:5). 
Regarding claim 18, Boncyk et al. discloses: 

A computer implemented method for image-based searching at a 
computer server, the method comprising: receiving an input image from a 
user device remotely located from the server (8:22-10:5); determining a 
plurality of graphical attributes represented in the input image (37:30-38:1 
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and 18:6-10); matching the input image to a reference image from a 
plurality of reference images stored in a storage medium, based on the 
plurality of graphical attributes of the input image and the reference 
images (15:1-15:35), each of the reference images having an associated 
symbolic identifier (37:30-38:1); and associating a symbolic identifier to 
the input image based on the symbolic identifier associated with the 
matching reference image (37:30-38:1); processing the symbolic identifier 
as search query to retrieve, from a search engine, a set of search results 
associated with the symbolic identifier (URL lookup 9:40-10:5); and 
transmitting a plurality of the search results to the user device (URL 
lookup 9:40-10:5). 

Regarding claim 19, see the rejection of claim 18 and note that Boncyk et al. 
further discloses: 

User device comprises a mobile telephone having an integrated 
camera (6:7-19). 

Regarding claim 21, see the rejection of claim 18 and note that Boncyk et al. 
further discloses: 

Receiving a geographic location of the user device through the 
network (27:36-28:14). 
Regarding claim 22, see the rejection of claim 21 and note that Boncyk et al. 
further discloses: 
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Image recognition system is further adapted to match the input 
image to a reference image from the plurality of reference images based 
on the geographic location of the user device (27:36-28:14). 
Regarding claim 23, see the rejection of claim 18 and note that Boncyk et al. 
further discloses: 

Image recognition system further includes a character recognition 
system (27:36-28:14). 
Regarding claim 24, see the rejection of claim 18 and note that Boncyk et al. 
further discloses: 

Image recognition system further includes a facial recognition 
system (27:7-9). 

Regarding claim 25, see the rejection of claim 18 and note that Boncyk et al. 
further discloses: 

Image recognition system is further adapted to enable transmission 
of reference images, for use by the image recognition system, to the 
storage medium (8:22-10:5). 
Regarding claim 26, see the rejection of claim 18 and note that Boncyk et al. 
further discloses: 

Selecting a matching reference image from a plurality of reference 
images stored in a storage medium comprises: determining the graphical 
attributes in the input image represented by a plurality of trained attribute 
detectors (9:32-10:2); aggregating a plurality of confidence values 
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received from the plurality of trained attribute detectors (9:32-10:2 and 
17:5-18:10); and determining the matching reference image where the 
aggregated plurality of confidence values exceed a predetermined 
threshold value (best match, 9:32-10:2). 
Regarding claim 27, see the rejection of claim 18 and note that Boncyk et al. 
further discloses: 

Search results comprise links to websites (9:32-10:5), contact 
information (36:18-21), product information (40:24-27), translations of 
recognized characters (27:36-28:14), and other information related to the 
input image (39:18-24). 
Regarding claim 28, Boncyk et al. discloses: 

A computer implemented method for image-based searching of 
product information (40:24-27), comprising: receiving an input image from 
a user device remotely located from the computer server (8:22-10:5); 
processing the input image of the manufactured product on an image 
recognition system to obtain a symbolic identifier identifying the 
manufactured product in the input image (symbols, 9:12-31), the symbolic 
identifier comprising at least one of a product name or a product 
identification number, or a product code (9:12-31); providing the symbolic 
identifier associated with the input image to the search engine as a query 
(decode symbol 9:12-31); receiving a set of search results associated with 
the symbolic identifier, the search results including at least one document 
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descriptive of the manufactured product in the input image (numerals or 
text information, 9:12-31, 40:24-27); and transmitting via the network, a 
plurality of the search results to the user device (37:30-38:1 ). 



Claim Rejections - 35 USC § 103 

Claims 9, 10, 16, and 20 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Boncyk et al. 

Regarding claim 9, see the rejection of claims 1 and 2 and note that Boncyk et al. 
further discloses in 40:24-27 that an image can be taken and used to retrieve 
maintenance instructions or repair history of a part of an aircraft. Official Notice is taken 
that it was well known at the time the invention was made to organize maintenance 
instructions for a part of a device into the form of a handbook, which would contain 
further maintenance instructions for other parts of the device. An advantage to doing so 
is that instructions for the device are organized for a person who desires to do further 
maintenance on different parts of the same device. For this reason it would have been 
obvious to one of ordinary skill in the art at the time the invention was made to have 
Boncyk include organizing maintenance instructions for a part of a device into the form 
of a handbook. 

It is noted by the examiner that because the applicant has failed to timely 
traverse the old and well-known statement above, it is now taken as admitted prior art. 
See MPEP 2144.03(c). 
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Regarding claim 10, see the rejection of claim 2 and note that Boncyk et al. is 
silent with regards to allowing providers of information to make new entries to the 
system so that their data can be retrieved. Official Notice is taken that it was well 
known at the time the invention was made to update databases with new information. 
An advantage to doing so is that the person deciding to access the database will have 
the most up to date information available. For this reason it would have been obvious to 
one of ordinary skill in the art at the time the invention was made to have Boncyk 
include allowing providers of information to make new entries to the system so that their 
data can be retrieved. 

It is noted by the examiner that because the applicant has failed to timely 
traverse the old and well-known statement above, it is now taken as admitted prior art. 
See MPEP 2144.03(c). 

Regarding claim 16, see the rejection of claim 1 1 and note that Boncyk et al. is 
silent with regards to updating the images and allowing future searching to include 
searching the updated images. Official Notice is taken that it was well known in the art 
at the time the invention was made to update image databases with new images that 
can be searched. An advantage to doing so is that the person deciding to access the 
database will have the most up to date information available. For this reason it would 
have been obvious to one of ordinary skill in the art at the time the invention was made 
to have Boncyk include allowing providers of information to make new entries to the 
system so that their data can be retrieved. 
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Regarding claim 20, see the rejection of claim 18 and note that Boncyk et al. is 
silent with regards to updating the images and allowing future searching to include 
searching the updated images. Official Notice is taken that it was well known in the art 
at the time the invention was made to update image databases with new images that 
can be searched. An advantage to doing so is that the person deciding to access the 
database will have the most up to date information available. For this reason it would 
have been obvious to one of ordinary skill in the art at the time the invention was made 
to have Boncyk include allowing providers of information to make new entries to the 
system so that their data can be retrieved. 

Claims 29-31 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Boncyk et al. in view of Waibel (U.S. Pub. No. 2003/0164819). 

Regarding claim 29, see the rejection of claim 28 and note that Boncyk et al. is 
silent with regards to using the method for identification of buildings name and 
information about buildings. Waibel discloses this in 1J0057 and 1J0061-0062. This is 
advantageous in that the user can annotate the images with the name and/or location 
for storage in a photo album. For this reason it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to have Boncyk et al. include 
using the method for identification of buildings name and information about buildings. 

Regarding claim 30, Boncyk discloses: 

A computer implemented method for image-based language 
recognition, comprising: receiving an input image from a user device 
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remotely located from the computer server (8:22-10:5); processing the 
input image on a character recognition system, executed by the computer 
server to obtain text data indicative of the text (27:36-28:14); and 
transmitting, via the network, the text to the user device (37:30-38:1). 
Boncyk et al. is silent with regards to inputting the text in one language and 
outputting a translation of the text in a second user language. Waibel discloses this in 
110028. This is advantageous in that a user can identify a building or read a sign that 
the user is interested in knowing information about. For this reason it would have been 
obvious to one of ordinary skill in the art at the time the invention was made to have 
Boncyk et al. include inputting the text in one language and outputting a translation of 
the text in a second user language. 

Regarding claim 31, see the rejection of claims 29, 11, and 15 and note that 
Boncyk et al. is silent with regards to providing the name and information about the 
name. Waibel discloses this in ^0058. This is advantageous in that the person's 
favorite foods, likes, and dislikes etc. can be ascertained. For this reason it would have 
been obvious to one of ordinary skill in the art at the time the invention was made to 
have Boncyk et al. include providing the name and information about the name. 

(10) Response to Argument 

Appellant argues that Boncyk does not disclose a search engine and only 
discloses a traditional database and point to cited section 9:40-10:5 of Boncyk. 
Appellant is reminded that during patent examination, the pending claims must be given 
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the broadest reasonable interpretation consistent with the specification. In re Morris, 
127 F.3d 1048, 1054, 44 USPQ2d 1023, 1027 (Fed. Cir. 1997). Additionally, although 
the claims are interpreted in light of the specification, limitations from the specification 
are not read into the claims. See In re Van Geuns, 988 F.2d 1 1 81 , 26 USPQ2d 1 057 
(Fed. Cir. 1993). 

The examiner further points out that 9:40-10:5 does disclose a URL for searching 
a network of computers or the internet and the URL lookup would provide information 
back to the user based on what the URL points to. However Boncyk is not limited to the 
URL application for a search engine. As can be seen in 27:36-28:17 identifying marks 
on a target object (100) from a camera can be included as object information from an 
identification server (106) with object recognition (107) and database (108) which is 
used by the content server (111, (search engine)) to identify an object from many similar 
objects. In 37:30-38:1 the relevant information can be sent back from the content 
server (111) after identifying the object. See also Figs. 5-7 of Boncyk. 

References for the subject matter pertaining to the Official Notices in the final 
office action are being provided as follows: 

For claim 9: The "Haynes Repair Manuals" document shows organized 
maintenance instructions for a part (automobile) into the form of a handbook (manual 
with instructions and photographs) where an automobile with many parts can be torn 
down and rebuilt using the instructions and photographs. 
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For claim 10: The Kaku reference in 1f0137 and H0139 shows the updating of an 
image database using captured images and searching and retrieving of images in the 
database using search parameters. 

For claim 16: The Kaku reference in 1f0137 and 1J0139 shows the updating of an 
image database using captured images and searching and retrieving of images in the 
database using search parameters. 

For claim 20: The Kaku reference in 1J0137 and U0139 shows the updating of an 
image database using captured images and searching and retrieving of images in the 
database using search parameters. 

(11) Related Proceeding(s) Appendix 

No decision rendered by a court or the Board is identified by the examiner in the 
Related Appeals and Interferences section of this examiner's answer. 

For the above reasons, it is believed that the rejections should be sustained. 
Respectfully submitted, 
Nicholas G Giles 
/Nicholas G Giles/ 
Examiner, Art Unit 2622 

Conferees: 



NgocYen Vu 



/Ngoc-Yen T. VU/ 
Supervisory Patent Examiner, Art Unit 2622 
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David Ometz 

/David L. Ometz/ 

Supervisory Patent Examiner, Art Unit 2622 



